Network monitoring method and apparatus

ABSTRACT

A network monitoring method and apparatus (61) is disclosed for monitoring communication connections temporarily established over a network (60) between respective pairs of entities for passing protocol data units therebetween. The connections are, for example, conducted in accordance with the TCP/IP protocol suite. The method involves the steps of monitoring the network to identify the protocol data units and the connection to which each such unit relates, and maintaining an active group (67) of call records each representing a respective connection considered to be currently active. The active group (67) of call records is maintained by adding a new call record to the group each time a protocol data unit is identified as relating to a connection unrepresented in the active group (67); updating an existing call record in the group (67) in response to any further protocol data units being identified as relevant to the connection represented by that record; and removing an existing call record from the active group to a completed-call group (69) when the corresponding connection is judged completed having regard to a continuing absence of further protocol data units relevant to that connection.

TECHNICAL FIELD

The present invention relates to a method and apparatus for monitoringcommunication connections established over a network and in particular,but not exclusively, to the generation of call records for connectionsconducted in accordance with the TCP/IP protocols.

BACKGROUND ART

Entities communicating with each other over a data communicationsnetwork generally do so by the exchange of data packets in accordancewith a predetermined protocol. Depending on the particular protocolused, the communication service provided between the entities willgenerally be either connectionless or connection-oriented. Aconnectionless service is one in which each packet is handled inisolation from any other packet, the service having no appreciation ofwhether or not the packet is one of a number of packets that togetherform a complete message. In contrast, a connection-oriented service willestablish a virtual circuit between entities that wish to communicate,in order to provide a reliable stream transport service for the packetspassed between the entities, the virtual circuit being closed when theentities have finished communicating with each other; such acommunication path established between entities over a virtual circuitis generally referred to as a connection whilst the communicationtransaction carried out from the setting up to the taking down of aconnection is often referred to as a call.

It is well known to provide network monitoring equipment both for thepurpose of traffic monitoring and for the purpose of fault analysis.Such monitoring equipment is mostly concerned either with analyzingindividual packets or with the aggregate effect of the monitored packetsas a whole (for example, for traffic estimation and network planning).U.S. patent specification No. 5,101,402 discloses a somewhat moresophisticated approach that provides for the collection of statistics onsessions conducted across connections. However, a drawback of the methoddisclosed in U.S. Pat. No. 5,101,402 is that it must track sessionprotocol interactions in order to ascertain when each sessionterminates; as a consequence, if the passage of the relevant protocolcommand terminating a session is missed for any reason (such as noise orpacket re-routing), an error condition will result.

It is an object of the present invention to provide a method andapparatus for monitoring calls over a network that is resilient topacket loss.

DISCLOSURE OF THE INVENTION

According to one aspect of the present invention, there is provided amethod of monitoring communication connections temporarily establishedover a network between respective pairs of entities for passing protocoldata units therebetween, each protocol data unit passed over the networkbeing provided by the sending entity with associated connectioninformation identifying the connection to which it relates, the methodinvolving the steps of monitoring the network to identify said protocoldata units and the connection to which each such unit relates, andmaintaining an active group of call records each representing arespective said connection considered to be currently active, saidmaintaining step involving:

adding a new call record to said active group each time a said protocoldata unit is identified as relating to a connection unrepresented insaid active group;

updating an existing call record in said active group in response to anyfurther said protocol data units being identified as relevant to theconnection represented by that record; and

removing an existing said call record from said active group when saidconnection is judged completed having regard to a continuing absence offurther protocol data units relevant to said connection.

Generally, the call records removed from the active group are retainedas a completed-call group of records thereby providing a historicalrecord of calls made. Furthermore, each call record will normally recordaggregated quantitative information (for example, number of data bytestransferred) for the protocol data units relevant to each direction ofdata flow between the pair of entities involved in the connection towhich the record relates.

The aforesaid associated connection information of each protocol dataunit may, for example, comprise the network addresses of each of theentities of the pair of entities associated with the connection to whichthe protocol data unit relates; in this case, the step of monitoring thenetwork preferably involves identifying the connection to which eachprotocol data unit relates by forming a connection identifier from thenetwork addresses contained in the associated connection information ina manner such that the connection identifier has a form that isindependent of the direction of passage of the protocol data unitbetween the entities, the connection identifier being used in saidmaintaining step to identify the corresponding call record.

To facilitate the removal of call records from the active group, eachcall record in the active group is preferably provided with a respectiveactivity indicator which is set when the record is created and when therecord is updated. The removal of call records from the active group isthen effected by checking the call records of the active group atintervals and each time removing those records whose activity indicatorsare in a reset state and resetting the activity indicators of theremaining call records.

Where at least some of the protocol data units have associated controlcodes relevant to the progression of the connections to which theyrelate, the step of monitoring the network preferably includesidentifying these control codes and storing them as part of theassociated call record; this then enables call fragments erroneouslyidentified as separate calls, to be subsequently pieced together byscanning the call records of the completed-call group for recordsrelating to connections between the same pair of entities, anddetermining from the control codes stored in such records whether therecords fit together as partial records of the same connection.

Preferably, each call record includes a start-of-call information itemset upon creation of that record to indicate a start time of thecorresponding connection, and an end-of-call information item which isupdated for each protocol data unit subsequently identified as relevantto that record to indicate a potential end time of the correspondingconnection. To this end, the step of monitoring the networkadvantageously involves associating a respective time stamp with eachsaid protocol data unit that is identified in that step; thestart-of-call information item of each call record is then set to thetime stamp of the protocol data unit causing the call record to becreated, and the end-of-call information item of the same record is setin turn to the time stamp of each successive protocol data unitidentified as relevant to the connection concerned.

Where at least some of said protocol data units have associated controlcodes relevant to the establishment of the connections to which theyrelate, each call record may include the identity of the entityinitiating the corresponding connection, the control codes associatedwith the protocol data unit causing a new call record to be added to theactive group serving to indicate which of the two communicating entitiesinvolved with a connection initiated the connection.

According to another aspect of the present invention, there is providedapparatus for monitoring communication connections temporarilyestablished over a network between respective pairs of entities forpassing protocol data units therebetween, each protocol data unit passedover the network being provided by the sending entity with associatedconnection information identifying the connection to which it relates,said apparatus comprising monitoring means for monitoring the network toidentify said protocol data units and the connection to which each suchunit relates, and call-record means connected to said monitoring meansand operative to maintain an active group of call records eachrepresenting a respective said connection considered to be currentlyactive, said call-record means comprising:

storage means for storing said active group of call records,

record-creation means for determining whether the said connection towhich each said protocol data unit relates, as identified by saidmonitoring means, is represented by a said call record in said activegroup, and where said connection is determined to be unrepresented by asaid call record, adding a new call record to said active group;

record-updating means for updating an existing call record in saidactive group in response to any further said protocol data units beingidentified by said monitoring means as relevant to the connectionrepresented by that record; and

record-removal means for removing an existing said call record from saidactive group when said connection is judged inactive having regard towhen the last protocol data unit relevant to said connection wasidentified.

The method and apparatus of the invention are particularly suited tocall record generation where communication is effected using the TCP/IPprotocol suite.

BRIEF DESCRIPTION OF THE DRAWINGS

A call record generator embodying the present invention and a networkmonitoring method in accordance the invention, will now be particularlydescribed, by way of non-limiting example, with reference to theaccompanying diagrammatic drawings, in which:

FIG. 1 is a diagram illustrating protocol stacks of two communicatingend systems;

FIG. 2 is a diagram illustrating the encapsulation of data transmittedbetween communicating entities in accordance with the TCP/IP protocolsuite;

FIG. 3 is a diagram illustrating the exchange of control messagesbetween end systems during the opening and closing of a TCP/IPconnection;

FIG. 4 is a diagram illustrating the main processes and data structuresemployed in the network monitoring method;

FIG. 5 illustrates the contents of a call record constructed during thenetwork monitoring method;

FIG. 6 is a flow diagram of a "Process Next Datagram" processillustrated in FIG. 4; and

FIG. 7 is a flow diagram of a "Completion Scan" process illustrated inFIG. 4.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 of the accompanying drawings shows conceptual protocol stacks 1and 2 operated by two end systems in communicating with each other overa network (12). Each protocol stack 1,2 is made up of a number ofdifferent layers each of which performs a particular task in thecommunication process. Considering by way of explanation, layer N ineach protocol stack 1,2, this layer N provides services to the layerabove (layer N+1) and in doing so utilizes services provided by thelayer below (layer N-1).

Within each layer N a protocol entity 3,4 controls the carrying out ofthe communication tasks assigned to that layer, this control beingeffected in coordination with the corresponding protocol entity of thecommunicating end system. Conceptually the protocol entities 3,4 in thesame protocol layer of the communicating end systems communicate andcoordinate with each other in accordance with a peer protocol (for layerN this is the layer N protocol shown in FIG. 1). The peer protocoldefines the form and sequencing of messages passed between the peerprotocol entities 3,4 in the form of protocol data units 5,6. Eachprotocol data unit (PDU) 5,6 contains protocol control information PCIand one or more service data units SDU, the latter being data which thelayer N protocol entity is handling on behalf of the layer N+1 above.

Whilst conceptually the peer protocol entities 3,4 are communicatingwith each other by passing protocol data units directly betweenthemselves, in practice, of course, the protocol data units must passdown one protocol stack, across the network 12 and up the other protocolstack to the relevant layer N. A protocol data unit passed by theprotocol entity of layer N down to layer N-1 is treated by that latterlayer as a service data unit SDU and handled appropriately.

Such conceptual layering of communication protocol stacks is well knownin the art--see, for example, the seven-layer OSI (Open SystemsInterconnect) model defined by the International Standards Organization(International Standard ISO7498). It will be appreciated that inpractice the protocol stacks are implemented primarily in software,although the lower levels may well be effected using dedicated hardware.

Each peer protocol may be connectionless or connection-oriented in form.If a peer protocol N is connectionless, the associated protocol entities3, 4 handle each service data unit SDU passed down to them from layerN+1 as an isolated item; in contrast, if a peer protocol N isconnection-oriented, the associated protocol entities 3,4 provide areliable stream transport service for service data units SDUs passeddown from the layer N+1. Generally, most of the peer protocols will beconnectionless with one or two key peer protocols being connectionoriented.

A protocol entity in layer N may be required to provide a service to anumber of different protocol entities in the layer N+1 above. Thisgenerally requires that an entity in layer N+1 of a first end system,when sending a protocol dam unit PDU to its peer entity in a second endsystem, provides adequate identification of the destination peer entityto the layer N service--providing entity of the first end system, sothat the peer entity in layer N of the second end system can forward thePDU to the appropriate entity in layer N+1.

A protocol entity in, for example, layer N+1 of one end system may alsobe required to communicate with peer entities in a number of other endsystems. In this case, where the layer N protocol isconnection-oriented, then, of course, it is inadequate for the layer Nprotocol entity to keep track of its current connections by simply byreference to the identity of the destination layer N+1 entity. Instead,connections are frequently identified by reference to the combination ofsource and destination entity identities.

The present invention relates to communications between entities, forexample in layer N+1, by the transfer of protocol data units using theservices of the layer-N protocol entities, where the layer N peerprotocol is connection-oriented. In the following description, the wellknown transmission control protocol TCP is used as an example of aconnection-oriented protocol, this protocol being used in conjunctionwith the Internet Protocol IP.

In TCP/IP networks, TCP-layer entities use pairings of endpoints toidentify a connection, where an endpoint is the combination of aparameter (IP address) identifying the end system concerned (or, moreaccurately, an interface of the end system to the network), and aparameter (TCP port number) indicating the source/destination endpointentity within the end system with which the TCP protocol entity is tocommunicate. A fuller explanation of TCP connection identification maybe found in a reference work such as "Internetworking with TCP/IP",Douglas E. Comer, Volume 1, Second Edition 1991, Prentice-Hall.

FIG. 2 illustrates how a protocol data unit (PDU) 20 is prepared fortransmission over a network from an endpoint entity A in one end systemto a peer endpoint entity B (not shown) in another end system inaccordance with the TCP/IP protocols, it being assumed that anappropriate TCP connection has already been established. As can be seen,the PDU 20 is passed down through three protocol layers (TCP layer 22,IP layer 23, and network interface layer 24) before being sent acrossthe network to the receiving end system. In each layer, an encapsulationprocess takes place as will be described more fully below; although notillustrated in FIG. 2 for reasons of simplicity, a fragmentation processmay also occur in one or more layers with the data received by the layerbeing split up into several units before being passed to the next layer.

As illustrated in FIG. 2, the basic protocol data unit of the TCP layerprotocol is the TCP segment 25 which comprises a TCP header 26 and a TCPdata area 27. The PDU 20 passed down from the endpoint entity A to theTCP layer 22, constitutes a service data unit for the TCP layer 22 andforms the TCP data area 27 of a TCP segment 25. The TCP header 26contains a number of information fields of which only those relevant tothe present invention are illustrated, these being a source port field28 holding the TCP port number of the endpoint entity A, a destinationport field 29 holding the TCP port number of the destination endpointentity B to which the PDU 20 is being sent, a sequence number field 30containing a sequence number for the current TCP segment 25 in relationto other such segments for the same connection, an acknowledgementnumber field 31 for containing the sequence number of the segment nextexpected from the peer TCP layer 22 in relation to a current connection,and a code field 32 containing various control codes.

Each TCP segment 35 is passed down to the IP layer 23 as a service dataunit of that layer. The basic protocol data unit of the IP layer 23 isthe IP datagram 35 comprising an IP header 36 and an IP data area 37.The IP data area 37 is occupied by the service data unit received fromthe TCP layer 22, that is by the TCP segment 25. The fields of the IPheader relevant to the present invention are a source IP address field38 indicating the IP address of the sending end system, and adestination IP address field 39 containing the IP address of thereceiving end system. The source and destination IP addresses are madeavailable to the TCP layer 22 to enable the latter to identifyconnections based on the pairings of IP address and TCP port number forthe source and destination endpoint entities.

In practice, there is frequently a one-to-one correspondence between TCPsegments and IP datagrams with no fragmentation of TCP segments; forsimplicity, such an arrangement is assumed hereinafter.

The network interface layer 24 serves to match the characteristics ofthe underlying physical transmission network to the higher-level layers.The network interface layer 24 is arranged to receive IP datagrams 35 asservice data units. The basic protocol data unit of the networkinterface layer 24 is a frame 45 which comprises a frame header 46 and aframe data area 47. The frame data area is occupied by the service dataunit received from the IP layer 23, that is by the IP datagram 35.

It will be appreciated that in the case illustrated in FIG. 2, eachframe 45 appearing on the physical network contains the IP address andTCP port number of the source and destination endpoint entities usingthe services of the TCP layer 22 to communicate with each other, theseparameters being sufficient to uniquely identify the TCP connectioninvolved. In practice, each IP datagram may be fragmented across severalframes; however, generally the frame carrying the first fragment of anIP datagram will be of sufficient size that this fragment includes notonly the datagram header, but also the TCP header of the TCP segmentencapsulated in the datagram; this is assumed to be the case for thepurposes of the following description.

As already discussed above, the TCP layer 22 provides aconnection-oriented communication service to the layer above, that is tothe endpoint entity A of FIG. 2. The overall progression of acommunication between endpoint entities A and B is therefore as follows.Upon the endpoint entity A requesting the services of the TCP layer 22,the latter establishes a connection with the peer TCP layer in the endsystem containing the endpoint entity B. Once the connection isestablished, PDUs 20 are passed between the endpoint entity A and itspeer entity B until both these entities have finished communicating.Thereafter, the TCP layer closes the connection. FIG. 3 illustrates thehandshakes involved in establishing and closing down a TCP connection.In this Figure, TCP peer entities in end systems 1 and 2 are considered.

The top half of FIG. 3 shows the handshakes involved in opening a TCPconnection. In the present example, the TCP layer entity in end system 1initiates opening of the connection by sending a segment 25 to its peerentity in end system 2 with a SYN bit set in the code bits field 32 ofthe TCP header 26 (arrow 50). On receiving this segment, the TCP entityin end system 2 responds with the segment in which SYN and ACK bits areset (arrow 51 ). Successful receipt of this reply by the TCP entity inend system 1 results in a concluding handshake segment being sent fromend system 1 to end system 2 in which the ACK bit in the code bits field32 is set (arrow 52); this serves to inform the end system 2 that bothsides have agreed that a connection has been established. During thishandshake process, sequence numbers are sent and acknowledged to enablesubsequent tracking of segments exchanged during reconnection. Thesesequence numbers and acknowledgement numbers are contained in fields 30and 31 of the TCP header 26.

The lower half of FIG. 3 illustrates the handshakes involved in closingdown a TCP connection. Closing a connection takes place in two stageswith the connection first being closed in one direction when the sendingTCP entity for that direction has no further data to communicate, andthen subsequently in the other direction when the other TCP entity hasfinished sending data. In the FIG. 3 example, it is assumed that the TCPlayer entity in end system 1 first wishes to close the connection and todo this it sends a segment 25 in which a FIN bit is set in the code bitsfield 32 of the TCP header 26 (arrow 53). The receipt of this segment isin due course acknowledged by the TCP entity in end system 2 by thereturn of a segment with the ACK bit set (arrow 54). The TCP entity inend system 2 then continues to send segments carrying data until no moredam remains to be communicated. At that time, the TCP entity in endsystem 2 sends a segment with the FIN and ACK bits set in the code bitsfield (arrow 55). Successful receipt of this segment by the TCP entityin end system 1 is acknowledged by a final segment 25 being sent withits ACK bit set (arrow 56).

It will be appreciated that in the course of any connection establishedbetween the TCP entities of end systems 1 and 2 there occurs apredetermined pattern of code bits in the code bit field 32 ofsuccessive segments as determined by the opening and closing handshakesillustrated in FIG. 3.

FIG. 4 illustrates the call record generator 61 embodying the presentinvention, the call record generator 61 being connected to monitor anetwork 60 in order to provide a record of each TCP connection (or call)temporarily established over the network. As can be seen, the callrecord generator 61 comprises a network interface unit 62 connected tothe network 60 and a processing sub-system 63 including a processor 65and memory 64. The memory 64 will generally comprise working RAM memory,ROM memory for program storage, and disk memory (these elements are notindividually shown in FIG. 4 for reasons of clarity).

The network interface unit 62 captures each frame 45 appearing on thenetwork and passes it to the processor sub-system 63 for processing.

As illustrated in FIG. 4, the processing sub-system 63 utilizes fourmain data structures 66-69 and executes four main processes 70-73.

More particularly, upon a frame being passed to the processingsub-system 63, the process 70 is initiated (for example as an interruptservice routine) to extract datagram information from the frame andstore the datagram, together with a time stamp, in a buffer constitutedby the data structure 66. The stored datagram information will generallyonly comprise the information of interest, namely the IP header and theheader of the encapsulated TCP segment; furthermore, if the datagram hasbeen subject to fragmentation, then only the first fragment, containingthe IP header and the header of the encapsulated TCP segment, isprocessed, all other fragments being discarded.

The process 71 which runs in background continuously monitors the buffer66 for new entries and whenever one or more entries are present,extracts the head entry for processing. As will be more fully describedhereinafter, this processing involves identifying the connection towhich the head entry damgram belongs and then updating a correspondingcall record for that connection, the call records for active connectionsbeing held in a control block table constituted by the data structure67. If a datagram corresponds to a connection for which no call recordexists in the control block table 67, then a new record is created bythe process 71. The process 71 also keeps a separate linked list ofactive call records, this list constituting the data structure 68. Eachcall record contains statistics on the corresponding TCP connection aswill be more fully described hereinafter with reference to FIG. 5.

At periodic intervals a completion scan process 72 is run which scansthe call records in the control block table 67 and removes any recordswhich have been inactive for a given minimum period (by "inactive" ishere meant that the record has not been updated in response to receiptof a further datagram associated with the connection concerned). Theprocess 72 uses the linked list 68 of call records to carry out thescanning of the call records. Any call records identified as inactive bythe process 72 are removed to a completed call record archiveconstituted by the data structure 69 (this archive is, for example, keptin disk storage or held off-line rather than being stored in RAM).

Finally, a call-record fragment assembly process 73 which is an offiineprocess, is used to examine the call records contained in the archive 69for records which in fact constitutes call record fragments relating tothe same connection, as will be explained more fully below.

A fundamental requirement of the overall call record generation processis, of course, that a unique connection identifier can be formed fromeach captured IP datagram with this identifier being the same regardlessof which of the two endpoint entities generated the captured datagram.In the present example, this unique connection identifier, which isgenerated by the process 71, is formed by the tuple of (IP address, TCport) of each endpoint entity with the numerically greater address plusport number always being placed first. For example, if an IP datagram iscaptured in which the source IP address is 15.8.61.123 and the source TCport is 123, whilst the destination IP address is 16.8.9.123 and thedestination TCP port is 111, then the resulting connection identifierwould be ((16.8.9.123,111),(15.8.61.123,123)). More generally, if theendpoint entity A has a four byte IP address made up of bytes aIp1 toaIP4 and a two byte TCP port number made up of bytes aPort1 and aPort2,and the endpoint entity B similarly has a four byte IP address made upof bytes bIP1 to bIP4 and a 2-byte TCP port number made up of bytesbPort1 and bPort2, then the connection identifier will be:

([aIP1.aIP2.aIP3.aIP4], [aPort1.aPort2]) ([bIP1.bIP2.bIP3.bIP4],[bPort1.bPort2]) where (aIP.aPort>bIP.bPort)

This connection identifier is, as already noted above, generated by theprocess 71. In fact, the connection identifier is not used directly toaccess the corresponding call record in the control block table 67 as,instead a hash key is created from the connection identifier and used tohash into the control block table 67. In the present embodiment the hashkey used is:

(aIP4,bPort2,aPort2,bIP4)

The contents of each call record are illustrated in FIG. 5. As can beseen each call record includes a first group of fields containing theconnection identifier for the connection to which the call recordrelates, a start time field corresponding to the time stamp associatedby the process 70 with the IP datagram giving rise to the generation ofthe call record, an end time field corresponding to the time stampassociated with the most recently received datagram used to update thecall record 75, a call origin field containing an indication of whichendpoint entity initiated the connection, and an activity flag fieldused in the determination of whether or not the record is still active.In addition to these general fields, each call record 75 includes twogroups of fields 77,78 relating to the statistics of the call in each ofthe two directions. Thus, in relation to endpoint entity A, the callrecord 75 includes fields containing the physical address of A (thisinformation is extracted from each frame received by the call recordgenerator and stored along with the datagram information by the process70), the IP address and TCP port number of endpoint entity A, the totalnumber of TCP bytes sent to endpoint A, the total number of TCP segmentssent to endpoint A, the total number of IP bytes sent to endpoint A, acumulative set of all TCP flags sent to A, the first sequence numbersent to A and the last sequence number sent to A. Corresponding fieldsare present in the record 75 for the endpoint entity B.

The reason sequence numbers are collected is to validate the totalnumber of TC bytes sent in a particular direction. For any givendirection, the second sequence number minus the first sequence numberwill give an approximate indication of the total number of TCP bytessent in that direction; this indication is only approximate becausethere are cases, namely ACK segments, where no TCP data is present butthe sequence number is incremented by 1. The set of all TCP flags seenis collected to facilitate call record fragment reassembly carried outby process 73.

A more detailed description regarding the generation and updating ofcall records by process 71 will now be given with reference to the flowdiagram illustrated in FIG. o 6. Once initiated, the process 71continually checks the buffer 66 (step 81) until it detects that adatagram with its associated time stamp has been entered into thebuffer. Thereupon the process 71 retrieves the head entry from thebuffer 66 (step 82) and constructs the connection identifier for thehead entry datagram (step 83). The process 71 also constructs thecorresponding hash key for the connection (step 84) and then attempts toaccess the control block table 67 using this hash key (step 85). An openhashing technique is used, that is, collision resolution is achieved bymaintaining separate overflow areas for each entry in the hash table 67.If no entry is found at the location indicated by the hash key, then itis assumed that the datagram being processed relates to a new connection(step 86); if, however, a matching entry is found at the locationindicated by the hash key, then it is assumed that the datagram relatesto the connection associated with the call record at that location.

Considering first the situation where a new connection is assumed, theprocess 71 now proceeds to create a new call record 75 in the controlblock table 67 (step 87) and to update the active record list 68 (step88). In creating the new call record, the process 71 enters theconnection identifier in the corresponding record field and also fillsin the start time and call origin fields (step 89). As already noted,the start time field, is used to hold the time stamp associated with thedatagram initiating call record generation. The call origin field of thecall record 75 contains an indication as to which end point entity ofthe connection originated the call. This information is inferred fromthe TCP code bits in the datagram initiating call record generation, itbeing appreciated that this first datagram will generally be oneinvolved in the establishment of the TCP connection concerned so thatthe code bits in the TCP header will follow the sequence illustrated inthe top half of FIG. 3. In particular, there are three main cases toconsider:

1. The TCP code bits contain only a SYN which implies that this is thefirst datagram for the connection so that the connection origin is theendpoint entity identified by the source address in the datagram;

2. The TCP code bits contain both a SYN and an ACK thereby indicatingthat the datagram relates to a response to a connection request so thatthe originating entity for the connection is the one identified by thedatagram's destination address;

3. The code bit flags do not contain a SYN; this indicates that theconnection establishment phase has been missed.

In the last of the three cases, the call origin field of the call record75 is set to contain an indication that the origin is unknown.

In generating a new call record 75, the process 71 also enters for eachendpoint entity, the physical address, IP address and TCP port number ofthat entity (step 90).

Further entries in a call record 75 are made by the process 71 in steps91 and 92, these entries being made both for a newly created call recordand for a call record corresponding to an existing connection. In step91, the end time field of the call record is set to the time stamp ofthe datagram being processed by process 71 (of course, for anewly-created call record, the end time will correspond to the starttime); in addition, the activity flag of the call record is set. In step92, statistics relating to the receiving endpoint entity are updated. Inparticular, the first sequence number is entered in the call record ifnot already present, the number of TCP bytes, TCP segments, and IP bytesare updated, the accumulated set of TCP flags is updated, and the lastsequence number seen is entered.

After completion of step 92, process 71 returns to checking for an entryto be processed in buffer 66 (step 81).

In this manner the process 71 builds up a record of each call conductedover the network.

The removal of calls corresponding to connections which have terminatedfrom the group of active call records in control block table 67 iseffected by the completion scan process 72. This process 72 isillustrated in flow chart form in FIG. 7. More particularly, at periodictime intervals, for example once every three minutes, set by aninterrupt timer, the process 72 is started (step 100) and a next-entrypointer is initialised to point to the head of the active call recordlist 68 (step 101). Thereafter, the list entry pointed to by thenext-entry pointer is fetched (step 102) and the next-entry pointer isupdated to point to the subsequent list entry, if any. Next, theactivity flag of the call record identified by the fetched list entry isexamined (step 103). If this activity flag is in a reset state thisindicates that the call record has not been created or updated since thelast running of the completion scan processor 72 (it being recalled thatthe activity flag is set by process 71 during creation/updating of acall record); these records are taken as relating to completedconnections and, accordingly, the process 72 proceeds to remove theserecords from the active call record group held in table 67 and transferthem to the completed call record archive 69 (step 104). Thereafter, theprocess 72 updates the active call list 68 by removing the entrycorresponding to the transferred call record.

In the event of the activity flag of the call record checked in step 103being in a set state, it is assumed that the corresponding connection isstill active and the call record is left in the control block table 67;however, its activity flag is reset (step 106).

After the completion of step 105 or 106 as appropriate, the process 72proceeds to step 108 in which it checks whether any more entries arepresent in the list 68 as indicated by the next-entry pointer. Iffurther entries are present, then the process loops back to step 102.However, if all entries in the list have been processed, process 72restarts the interrupt timer used to time the intervals betweensuccessive scans by the process 72 (step 109) and then terminates (step110).

It will be appreciated that if an existing call record is not updated inthe period between successive scans, then it will be removed from theactive call group held in the control block table 67 during the runningof the later of the completion scan process 72. In the present example,the interval between successive runs of the process 72 is three minutes.However, other intervals are possible although the intervals should belonger than 45 seconds as this is the interval that TCP keep-alivepackets are sent on certain types of TCP connection. The choice ofinterval comes down to a trade-off between using up space in the table67 for idle/complete connections, the amount of time required to scanthe active call group, and the amount of off-line call record fragmentreassembly that needs to be done to build up complete call records frompreviously generated incomplete ones (the process 73 illustrated in FIG.4). The advantages of the above-described process 72 is that it obviatesthe need to check all datagrams to see if it is the last one to be sentfor a particular TCP connection and this reduces the processing time perdatagram. Furthermore, the process 72 is still effective even if thedatagrams containing the connection close-down sequence are lost.

Of course, if a connection should be idle longer than the intervalbetween successive runnings of the completion-scan process 72, thecorresponding call record will be removed from the table 67 before theconnection is complete. The remainder of the connection transaction willthen be logged as a separate connection with its own call record beingcreated. As a consequence, a number of partially complete call recordsbetween the same endpoint entities may be generated. However, thecall-record fragment assembly process 73 is used to reassemble thesecall record fragments by reference to the code bits noted in the TCPflag field of each call record. More particularly, by using the TCP flaglist and the start and end times, the process 73 pieces together callrecord fragments into a set of complete call records. For example, acall record might be split into three calls and the TCP flag list ofeach of the call records might be as follows:

Record 1 - SYN, ACK, PUSH

Record 2 - ACK, PUSH

Record 3 - ACK, PUSH, FIN

Given that a normal call should have a flag list containing a SYN, ACK,and FIN (see FIG. 3) and generally also PUSH, then it is fairly certainthat the above three call records are for the same connection (assumingthat the start and finish times match appropriately). This process, doesnot, of course, guarantee that every call fragment will be picked upsuccessfully, but the majority will.

It will be appreciated that many variations are possible to theabove-call record generator and monitoring method. In particular, itwill be appreciated that the call record generator and monitoring methodcan be applied to monitoring of connections other than TCP connections.Furthermore, a filter could be incorporated into the call recordgenerator to filter out certain categories of frames so that, forexample, the call record generator only generated records in respect ofcalls sourced from a particular network node or sub-network. With regardto the detailed implementation of the call record generator, onepossible variant would be for the completion-scan process 72 to use theend time field of each call record to judge whether the correspondingconnection is still active. In this case, the contents of the end timefield would be compared to the current time as provided by the timestamp timing source. Such an arrangement would obviate the need for anactivity flag field. Another possible variant would be to store both theactive and completed call records in the same memory and distinguish themembers of each group from each other by an appropriate flag associatedwith each record. Furthermore, it will be appreciated that it is notessential to use a hash table for accessing the active call records.

I claim:
 1. A method of monitoring communication connections temporarilyestablished over a network for passing protocol data units between atleast several entities, each protocol data unit passed over the networkbeing provided by a sending entity with associated connectioninformation identifying the connection to which it relates, the methodcomprising the steps of monitoring the network to identify said protocoldata units and the connection to which each such unit relates, andmaintaining an active group of call records each representing arespective said connection considered to be currently active, saidmaintaining step comprising:adding a new call record to said activegroup each time one of said protocol data units is identified by themonitoring step as relating to a connection that is not represented insaid active group; updating an existing call record in said active groupin response to any further of said protocol data units being identifiedby the monitoring step as relevant to the connection represented by saidexisting call record; detecting that said connection has been completed,the detecting step being performed with regard to a continuing absenceof further protocol data units relevant to said connection; and removingseveral ones of said existing call records from said active group whensaid connection has been detected as being completed.
 2. A methodaccording to claim 1, wherein each call record in said active group isprovided with a respective activity indicator which is set when therecord is created and when the record is updated, the removal of callrecords from said active group being effected by checking the callrecords of the active group at intervals and each time removing thoserecords whose activity indicators are in a reset state and resetting theactivity indicators of the other records to a previous state.
 3. Amethod according to claim 1, wherein the call records removed from saidactive group are retained as a completed-call group of records.
 4. Amethod according to claim 3, wherein at least some of said protocol dataunits have associated control codes relevant to the progression of theconnections to which they relate, said step of monitoring the networkincluding identifying said control codes associated with said protocoldata units, and said maintaining step further including, for each saidcall record of said active group, storing as part of that record thecontrol codes relevant to the connection represented by said record;said method further including the step of scanning the call records ofsaid completed-call group for records relating to connections betweenthe same pair of entities, determining from the control codes stored insuch records whether the records constitute partial records of the sameconnection, and combining said records in response to the records beingdetermined as constituting partial records of the same connection.
 5. Amethod according to claim 1, wherein each said call record of saidactive group includes a start-of-call information item set upon creationof that record to indicate a start time of the corresponding connection,and an end-of-call information item which is updated for each protocoldata unit subsequently identified as relevant to that record to indicatea potential end time of the corresponding connection.
 6. A methodaccording to claim 5, further including providing a time stamp when eachprotocol data unit is identified, wherein said step of monitoring thenetwork includes associating a respective one of the time stamps witheach of said protocol data units that is identified in that step; saidstart-of-call information item of each said call record being set to thetime stamp of the protocol data unit causing the call record to becreated, and the end-of-call information item of the same record beingset in turn to the time stamp of each successive protocol data unitidentified as relevant to the connection represented by that record. 7.A method according to claim 1, wherein at least some of said protocoldata units have associated control codes relevant to the establishmentof the connections to which they relate, said step of monitoring thenetwork including identifying said control codes associated with saidprotocol data units, and said maintaining step further involving, foreach said call record added to said active group, determining from thesaid control codes relevant to the connection represented by thatrecord, which entity of said pair of entities associated with thatconnection initiated the connection, the identity of the initiatingentity being stored in said call record.
 8. A method according to claim1, wherein said step of monitoring the network further involvesdetermining for each identified said protocol data unit, bothquantitative information related to that protocol data unit, and inwhich direction the unit is being passed between the said pair ofentities associated with the said connection to which the protocol dataunit relates; said maintaining step being such that each said callrecord contains respective aggregated said quantitative information forthe protocol data units relevant to each said direction between saidpair of entities.
 9. A method according to claim 1, wherein saidassociated connection information of each protocol data unit comprisesthe network addresses of each of the entities of said pair of entitiesassociated with the connection to which the protocol data unit relates,said step of monitoring the network involving identifying the saidconnection to which each said protocol data unit relates by forming aconnection identifier from the network addresses contained in saidassociated connection information in a manner such that said connectionidentifier has a form that is independent of the direction of passage ofthe protocol data unit between said entities, said connection identifierbeing used in said maintaining step to identify the corresponding saidcall record.
 10. A method according to claim 9, wherein said maintainingstep involves storing the said call records of said active group in ahash table with the call record relevant to a particular said protocoldata unit being accessed in said hash table by use of a hash key formedfrom the said connection identifier associated with that protocol dataunit.
 11. Apparatus for monitoring communication connections temporarilyestablished over a network for passing protocol data units between atleast several entities, each protocol data unit passed over the networkbeing provided by the sending entity with associated connectioninformation identifying the connection to which it relates, saidapparatus comprising monitoring means for monitoring the network toidentify said protocol data units and the connection to which each suchunit relates, and call-record means connected to said monitoring meansand operative to maintain an active group of call records eachrepresenting a respective said connection considered to be currentlyactive, said call-record means comprising:storage means for storing saidactive group of call records, record-creation means for determiningwhether the said connection to which each said protocol data unitrelates, as identified by said monitoring means, is represented by asaid call record in said active group, and where said connection isdetermined to be unrepresented by a said call record, adding a new callrecord to said active group; means for detecting when said connectinghas been completed, the connection being detected as being completedwith regard to a continuing absence of further protocol data unitsrelevant to said connection, record-updating means for modifying saidstorage means so an existing call record in said active group is updatedin response to any further said protocol data units being identified bysaid monitoring means as relevant to the connection represented by thatrecord; and record-removal means for modifying said storage means so anexisting call record is removed from said active group when said meansfor detecting detects that the connection is completed.